{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "committed-brass",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy import stats\n",
    "import pymysql\n",
    "import warnings\n",
    "from sqlalchemy import create_engine\n",
    "warnings.filterwarnings('ignore')\n",
    "plt.rcParams['font.family']='KaiTi'\n",
    "plt.rcParams['axes.unicode_minus']= False"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "distant-transition",
   "metadata": {},
   "source": [
    "### 检验指标的确定"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "regulated-latitude",
   "metadata": {},
   "source": [
    "banner设计目的为了提升点击量，进而提升下单量\n",
    "\n",
    "一类指标：曝光时长，保证颜色变化对用户的体验或使用不产生影响，指标不下降或在正常范围内波动。\n",
    "\n",
    "二类指标：点击率，颜色的变化会吸引用户进行点击。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "concerned-proceeding",
   "metadata": {},
   "source": [
    "### 确定检验统计量"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "brief-webmaster",
   "metadata": {},
   "source": [
    "一类指标统计量：曝光时长的均值之差\n",
    "\n",
    "二类指标统计量：点击率的比例之差"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "applicable-sierra",
   "metadata": {},
   "source": [
    "### 数据埋点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "difficult-amazon",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>属性名</th>\n",
       "      <th>属性说明</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>page_id</td>\n",
       "      <td>页面id</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ad_id</td>\n",
       "      <td>广告位id</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>user_id</td>\n",
       "      <td>用户id</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>user_group</td>\n",
       "      <td>用户组别</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>session_start</td>\n",
       "      <td>进入时间</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>session_end</td>\n",
       "      <td>离开时间</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             属性名   属性说明\n",
       "0        page_id   页面id\n",
       "1          ad_id  广告位id\n",
       "2        user_id   用户id\n",
       "3     user_group   用户组别\n",
       "4  session_start   进入时间\n",
       "5    session_end   离开时间"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>属性名</th>\n",
       "      <th>属性说明</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>page_id</td>\n",
       "      <td>页面id</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ad_id</td>\n",
       "      <td>广告位id</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>user_id</td>\n",
       "      <td>用户id</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>user_group</td>\n",
       "      <td>用户组别</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          属性名   属性说明\n",
       "0     page_id   页面id\n",
       "1       ad_id  广告位id\n",
       "2     user_id   用户id\n",
       "3  user_group   用户组别"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 曝光事件 ad_view\n",
    "ad_view = pd.DataFrame({'属性名':['page_id','ad_id','user_id','user_group','session_start','session_end'],\n",
    "                        '属性说明':['页面id','广告位id','用户id','用户组别','进入时间','离开时间']})\n",
    "\n",
    "# 点击事件 ad_clicks\n",
    "ad_clicks = pd.DataFrame({'属性名':['page_id','ad_id','user_id','user_group'],\n",
    "                          '属性说明':['页面id','广告位id','用户id','用户组别']})\n",
    "\n",
    "display(ad_view,ad_clicks)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "elementary-threshold",
   "metadata": {},
   "source": [
    "### 确定H0,H1,显著水平α"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "incorrect-quantity",
   "metadata": {},
   "source": [
    "一类指标\n",
    "H0：对照组pv - 实验组pv ＜ 2 * 对照组pv标准差\n",
    "H1：对照组pv - 实验组pv ≥ 2 * 对照组pv标准差\n",
    "\n",
    "二类指标\n",
    "H0：对照组点击率 - 实验组点击率 ≤ 0\n",
    "H1：对照组点击率 - 实验组点击率 ＞ 0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "spiritual-gibraltar",
   "metadata": {},
   "source": [
    "α：0.05"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "criminal-bedroom",
   "metadata": {},
   "source": [
    "### 计算样本量"
   ]
  },
  {
   "attachments": {
    "image-3.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvwAAABKCAYAAADHVIDOAAAgAElEQVR4Aex9h1scR77t/X/eu/fu27139+5e73rXtmzJtnKwkq1kBSvnnHMWEkgCESQQOeecBIgMIuech5lhcg7nfdXDwMzQ3RMoEPiD72ump35Vp06dqu6prv5V1b9hHv7MZjPVXGjjUSXHA0abN208HupUTbR5u4InFAohkUiclsMVLKcgv4MIi1UH2rw9xRsaGoJGo/lsLcFT3lyEPcUbHByEVqu1g/UUyw7E5gsfHp/NBmLOTwcGBuZchzkvBIUMiA46nc4OaaHUkR2pefjS398PvV5vlxNNLWhiEZJ8eHw2uwKyfOnr65uhA0s0j4Nmw40tU9p41jyIDgaDwfqV+ifh/W/k31wfJpOJah608ea6/FZ82rxp41l5zvUnbd6u4I2Pj2NiYsJpO3QFa671WQj4i1UH2rw9xSMdXbVa7bS9zVVde8qbi4+neKSDRx58bHE9xbLFsD3nw+Oz2WLM9Tnp3M21DnNdBhr45EGYdPhtsRZKHdlymo9zogXp8NvmRVMLmliEIx8en822fGzn5F7pqANbPE/DZsONLU/aeNY8iA6kw2/9TvuT8P436o8RLICEOM0/2ng0ufFh0eZNG4+PO00bbd6u4C2N8LtXg65o6h7i/MSmzdtTvKURfkt9L43wW3RYGuHnvv49vca4ERevhaYWNLGIonx4fLbPXRu0udHGmy99CO+lDv98qe3kgvGExmJueJ6UlyuNKzosdfi51GMPd0VT9pSfN5Q2b0/xljr8lnaw1OG36LDU4ee+L3h6jXEjLl4LTS1oYhFF+fD4bJ+7Nmhzo403X/oQ3ksd/vlS28kF4wmNxdzwPCkvVxpXdFjq8HOpxx7uiqbsKT9vKG3enuItdfgt7WCpw2/RYanDz31f8PQa40ZcvBaaWtDEIory4fHZPndt0OZGG2++9CG8l3z452EOAxGaHLR9v2jjWXnO9Sdt3q7gLfnwuzdXxxVN57qdeIJPm7eneKSju+TDb8aSD7/lulvy4ee+/3h6jXlyf1joaWhqQROL6MaHx2f73JrT5kYbb770IbyXRvjn6/HKyROyJzRIQ1mMf7R5u4K3NMLvXktxRVP3EOcnNm3enuItjfBb6ntphN+iw9IIP/f17+k1xo24eC00taCJRRTlw+Ozfe7aoM2NNt586UN4L3X450ttJxeMJzQWc8PzpLxcaVzRYanDz6Uee7grmrKn/LyhtHl7irfU4be0g6UOv0WHpQ4/933B02uMG3HxWmhqQROLKMqHx2f73LVBmxttvPnSh/Be6vDPl9pOLhhPaCzmhudJebnSuKLDUoefSz32cFc0ZU/5eUNp8/YUb6nDb2kHSx1+iw5LHX7u+4Kn1xg34uK10NSCJhZRlA+Pz/a5a4M2N9p486UP4b3kw7/kw89cyKQxzNdB2wfOFTyxWAyZTOa0jK5gzZdOnzOfxaoDbd6e4o2OjjIbLX2uOvSUNxdfT/FGRkZm6OAplifcaOfFxcFZ+PDw8Jzr4IzDQrUvlDpaCPrQ1IImFtGGD4/P9rl1pc2NNt586UN4L43wz9fjlZMnZE9okIayGP9o86aJRxNrMdaNlfNi1YE2b9p4Vn3n+pM2b5p4NLGIjnx4fLa5rgNn+AuZmzPuNO1LOkyrSVMLmliEIR8en226dJ/njDY32njzpQrhvdThny+1nVwwntBYzA3Pk/JypaGpA00sLr6LIXyx6kCbN228+ap72rxp4tHEInry4fHZ5qsuuPJZyNy4OM9F+JIO06rS1IImFmHIh8dnmy7d5zmjzY023nypQngvdfjnS20nF4wnNBZzw/OkvFxpaOpAE4uL72IIX6w60OZNG2++6p42b5p4NLGInnx4fLb5qguufBYyNy7OcxHutg5mE3RqJZQqNYyL9C03l45ua8EF5OS64EnGaeLjxmfjBJwnA21utPEcZTAa9FDIFdAbjI6mWX0nvJd8+H/nvuukkhfaQdsHjiYeTayFprs7fBarDrR508Zzpw5mE5c2b5p4NLGIRnx4fLbZ6Esj7ULmRqN8rmK4q4NwqAPvfP1Q0tjHdPhdzWcxxHNXC74y0cQi+fDh8dn4OM6HjTY32niOGhg1UuQnhOBdQj7kWgO1/hvhvTTCP6tnJvcSk4ql+UcbjyY3PizavGni0cTi02Ch2xarDrR508abr3qnzZsmHk0soicfHp9tvuqCK5+FzI2L81yEu6ODQSnE22fX8TI8C1IN3RHQuSibu5juaOEMmyYWyYsPj8/mjOdc22lzo43HVn7pUDOeXLuMiIJG6Cg1c8J7qcPPpvYchdFuKLTx5qjYM2Bp86aJRxNrRsEXUcBi1YE2b9p489UEaPOmiUcTi+jJh8dnm6+64MpnIXPj4jwX4S7rYFSjNNYbR84/QY9QNRdUPjumy1q4wJQmFsmOD4/P5gLVOY1CmxttPPbCG9BblYzDh8+irF3AHsXNUMJ7qcPvpmiziU67odDGm03Z3ElLmzdNPJpY7miy0OIuVh1o86aNN1/1TJs3TTyaWERPPjw+23zVBVc+C5kbF+e5CHdNBzOGmwpxaOcuJFb2w0T3ZflcFMsjTNe0cA2aJhbJkQ+Pz+Ya27mLRZsbbTyukpv1KoQ/OYszd/0xLNVzRXM5nPBe8uGfRx932r5ftPFIg5iPgzZvmng0seZDy7nKY7HqQJs3bby5qi9HXNq8aeLRxCLl5sPjszlqNt/fFzK3+dTCFR3UEwPwuXIEB24EQGcwzsvv1HxqYM3LFS2scZ190sQiefHh8dmc8ZxrO21utPH4yi/oLMWh3TsQlFoOjd40q3ZPeC+N8Lv8fDT7iKRiaf7RxqPJjQ+LNm+aeDSx+DRY6LbFqgNt3rTx5qveafOmiUcTi+jJh8dnm6+64MpnIXPj4jwX4c50MBl1qMuPxvatu5BY2goT5d/RuSiTp5jOtHAHlyYWyZcPj8/mDue5iEubG208vjLrVBJEPDuP3cdvo31Ugtk0fcJ7qcPPpzZlG+2GQhuPcnE54WjzpolHE4tTgEVgWKw60OZNG2++qp42b5p4NLGInnx4fLb5qguufBYyNy7OcxHuTAe1TIDX137D7tOP0D2uAN1hs7kokeeYzrRwB5kmFsmXD4/P5g7nuYhLmxttPL4ym00GtJQmYcfmzQjKrIFxFr5shPdSh59Pbco22g2FNh7l4nLC0eZNE48mFqcAi8CwWHWgzZs23nxVPW3eNPFoYhE9+fD4bPNVF1z5LGRuXJznItyZDiOthfj5xx/xNCIPSoNpLigsGExnWrhDlCYWyZcPj8/mDue5iEubG208/jKboRhtx53j27Ht3DPI9Z4v2UN4L3X4+dWmaqXdUGjjUS0sDxht3jTxaGLxSLDgTYtVB9q8aePNV8XT5k0TjyYW0ZMPj8/mdl2YjZCIhlBfXYmPZRXoHByFfpYjbm5z+B0m4K0jswFJXqfw3bqd+NA4MCuXhsUgHa8WbhaAJhbJmg+Pz+YmberRaXOjjeeswGajCjnvH+HLf67Bhw6Rs+icdsJ7qcPPKQ99A+2GQhuPfonZEWnzpolHE4u99IsjdLHqQJs3bbz5qn3avGni0cQievLh8dncrQuDYhD+j+8gLCUfqWGvcPjoRdT1y9yFmYpPk9sU6CI84dPBIO/Hqa3Lse34Q/SK1IuwdO5R5tPCPST+68JdLBKfjxufzZO8aKahzY02nitl7atOxdpvvsDNtwXwdL0ewnupw++K2pTi0G4otPEoFdMpjDPeZpMJaoUcKo3OJX9NZ3hchDSSUdQ1dkJt85bYUyyuPOYm3ARBfyta+wWwoU41q8Whw8wi0+btMp7ZBJVcyqykMJMVhRCTHjKJFDqjax7MLvN2kRpNPJpYhD4fHp/NqFNDrlC5vMSjbqIXAU8eoahjHIqBCuz6YSXeFXa4qODMaHzcZsb+/Ybw6TDakIWVX/wNV17FQ6Z3re0vKKXMJsjFw+jo6IEr+4TxaeFuuWhikbzt8Yxor6/HmETD0LK3uct0buPT5kYbz5XSK8dacW7bcmw+dh9DSs/cegjvpQ6/K2pTikO7odDGo1RMpzDOeAt7P+GtfxDqe4UudWid4bER0kqHEBP0CvFFLbDtQxEs8sBhZJZ+Y0u5EMLMkI024dXTZyhvH3Ppochd1p5o6m4ecxGfNm+X8cx61GVHICA6H1I95ccwMnGrKBGvQ9IgUrvW6XGZt4uVQBOPJhahz4fHZ9OIOhHy+hVK2wQuXUNkAp1cSh7qtGjKC8PWnSdQN6RwUcGZ0fi4zYz9+w3h1sGEssi7+PMX3yEoo9Gl34IFp5LZhIHmQkRFJUFs6RvzUuTWgjcZq5EmFsnAHk+DyBcvUdEhZvK2t7HS+WyBtLnRxnNFGJNGBP+bu/HF97vxsc0ztx7Ce6nD74ralOLQbii08SgV0ykMF2+zUY/BpiLcu3YN+fWDzPJrrnRvuPDYiZihVwiRHOiFB+9SIFPrmGh6jQKDvV0oL87Goxsn4RWahAkXbtDsecx9KClzV3kKLt18iLpekcujlK4yc09TV1HnPh5t3q7jmWHSyxD7+j6eBiZgaEJFwd/YDL1aipLUYFx75IcBicZlTNd5u1Yn7uCZ9Ap8qqyDUMn+8tkdLFfY8eE5swnaSnDnxi1kVLRBa3A+ckaWiOxtLkeAfyDKGnqhUrv2FpKtHHzc2OL/XsO4dDBrBfA+sxtffr8RBZ3SxVl8swn9jXkIC4uHyIXfEy4tPCk8TSySvz2eBqFPvFDabul82ts8YTt3aWhzo43nUsnNehSEPMLf/udLBGRU2w1SupR+sv6WOvyuqkUhHu2GQhuPQhFdgmDlbTZgrKMcd65cQERhPQwuIVkiseJxpDebjej9lIcLZy+hqIU8VFgiKsSDyEpPQXiwLzav/Ccu+kS4NCLDkc28BKulYwh4cAleYdmQa513Vtwh5Y6m7uDOdVzavN3CM5shE/bA/9ktvA7PgkTF3uF1VQOjQYva3Ghcu/YQpW2jbq0/7hZvFwi5g2dQDODptWeoH1GyIruDxQrgEMiHx2cjMCaDDo1Fcbhx8z6KGwecLHtnQndDMQLevkN+ZS0qM9Lxsa7f45FnZ9wcivm7/cqlg2rkE47v/BHfbzqNTqk7vwgLSCrS4W/KR3h4wu+uwx/29DnKJieRctXhQqgJ2txo47mqUc+HUPzzb3/BpZfJUBhcGQq1Rya8lzr89prM6TfaDYU23pwW3gZ8Jm8zFKJ+BD6+hnt+CZBo3OsozcSzyczhlHSiMt7ewfHrvhiwecdqNptgMBohG23FiV9W4OLLSIi1DokX2FeTQY2SRF8cuvAQ3WMyl9wSXC2CO5q6ijkf8WjzdhePuH301+XhyunzSPjYOqsiizorcOPsKYRn1UDN4yakV02gKi8JAe+i0NQ3DsV4L4pLqu3mpsyKyIzRPX40o3IIL248R4OLHX6TyYDhzk+IDn2HhJxKyNUqdNVXo6V3zKUHf7464rNZSmGGXiVG0tsnOHPnNUak3MOwesUI/K6ewN4jJ3H1xhUcPXIJxS0CfjF4rM658ST+HZm4dBipy8GulV9g3dEXEC9G/31SRw4dfuIuKhWNYmB4DBq9HnLRKLq7+yBVapj7N5cWnlQ3H5ZeI0Nfby8kSg20KhkGeroxLJjgfXi1x9NgqcPvSa14nkbemYNlX/4Fv13ywYjC/QE+Un//Rv7N9UF7K2LaeHNdfis+bd608aw85/rTkbfJoEV58hscPnkN1R1jFleeGe1Sj5GuT4iLikRlyyBEI73ISY1DZFw6hie0LrdhnXoCXse34c67VMh0M9s+6fAf/3k5LvhEQKSZaZ+9NkaIBtuQEheFvOoOTAiHUZSZhMjYFHSMkJ303MnTiK7KTBzZeQg5jf0curmDNx3XsY7c4zWNM9/paPP2BM+oVyA35CG2H7yFfpnezTqd1M4gR+i94zh95y2GFTwYejlKEgJw9sx5nD97GucvXsetO/cRX1gPo1ttib/O3NGBrGbDdPiHFaxld8RSCtrx7OJJnD5zBseOncadhw/wwi8U7SMy1vSObcoRz9ZObFqNCgqlGgYj99b0E/21uHxwB57FV3DmaTJoMNLfh57eXvT29qC7bwgqnZEzvi0PtnM+3mzxf69h7DoY0JQfgzV//zN2PwyHhmJbnlcdTUb0Tbr0CNVmqETdCH/jg9u37uLB05cIi05AQmQIXrwOQefoBNi14L82ucrDhWXSSlCcHofAV09w+tJ9BIVGIiMtGc8fPGRc2/QcWtvjqSddeoRM+7e3ecaXqxyzDafNjTaeq+UziRqxYdnfsfnwVbSNqd2+7xDeSyP8nj9wuZ2SVCzNP9p4NLnxYTnylgw14PiurXgR+xFqtgdXswlj3fV4/z4Ycf4P8MOXK3H9RShq2rpQGHIHPx+8hqZR1YwRbrNBA5FgHLbeLmpJG47/tBkBKR+hZakO+Vib0xF+8jZAr9NCrVbzHhqNdspliNHDbIZ0rAfRIW8RG+KN5V+vwrl7L1FW34GKpDf4eccRlPfP3EnSbNBBPD7OMsprxnhrBc7v3Y6A3BbmBsCnuzs2xzpyJ+3njEubt6d4GnE7ru1ajRtv86Fx9/Wr2YiOgmBsXLsLHzokvHIatAr09XRDrFBDPNKNuNBARGaVQ6E1QKeWYaCrGdlpiegSeL58JCHAqYPJAIlYgMGhIQxNHgOdNbh37i4KajunwoaGhjEukkI/2em2LZRSMoKu3mGoNEq0V2fjmbcfmkfkMBt1kEwI0VheiNyCT1BwvPjj4kY66B0N5UhNS8X7oABEpJZAyfn20ICGdH+s/+k3lPTIeEc6bbnP5pyL92wwF2NaVh2MGuTHv8Q//vIHXA/KnJf6mBPt7Eb4zWjMi0Zk+gc8ubAP+y88x5hCA2F3Hc7s+RleUcXQOmwsNtxahriENIzbLiVnQ1Q43IWmpiY0tzTPOJqam9Dc2o5xidxGPzMEnTWISC5AeYoPVvy4DVk13TCYzUh9dgpbD95BD5lsQN4eZ0bg9vVrePXmLSIjIxAUEIDs0gYodeRHemmE36Ya5ufUMIIT33+Fr7cdxcf2cbfzJNfZUoffbdk8T8B6Y/McjvtHeBaY85HUXgc9isPuYNOus6jtY+/cMG4S7bUoLq9A1rubWL76V3zsFjM3scFifyz7ejViPnTZ3NRID8WAwbosXL18A62i6VVTlCPV2L5xO95n1LOuZ+tKh9+olaOjoQw52dnIzs5BTg7LkZWNgqJqu04KeVAQDrcjv7AExXEvsOzbjcisH2J4j9amY9OyZXiZ0Ai7Zx6zEeMd5bh38TwqBmeuQz3RW4Nzh7fjbkIV1fZgX0fz0Sro5EGbt8d4Zh3Kox9i7bZDKOuamPEwyldanbQPT0/vxOH7EXbthzONWY+x3mbERUcgr6IRco0BpK1NDHciKew19u3egszmYc7krAazGSaTEUbj9Ag2azydAvUVBYiPS0BiQiISE5KQGPUOh3Yfxsvg6MmwBCTEJ6GwrBkyleVthR2W2Qy1TIiqwnTExiejWyBnJicb1RI01JbB+8JRnLkShDHL/Hq7pOQLex0ZMdpWgfjUQojVenSUZeKW13uMS2deQ1ZAw0QnLuxcj4uvUiBlOjVWy9x8svOem7wWMiqbDmadHOnBN/C3P/0XnkWXLmT6/NzsOvwmSIVj6Gj8iCO7diAovQI6gxF9DR+wa/0GPAkvhNZmJ1WdXAC/G4ew99Q1dIrZn3b72yqQn5ePgvyCGQcJL/zwEb0jIpvfFDPUChkkchFCbh7AznMvMEKepA0KvDq/B5v23UbXuOUaGWzMwaG9R5FX2wOlSoUPUU+w99gNNA/Jlzr8/LU+R1Yprm9dhi9/2IOC6j63flMIIXKdLXX456hq2GDZbmxs8VwNo43nar6zjWfLWy9ux8Xd63DBOxFS3XTH3DYPEl+n1UAp7ofP8c3YejEAUjJqataiMPgO/vH1T0ipHLDp8JshFw3g/ZMz+HLNdpT0T//IqwR12LdpO4LTP4Gt/+BKh5+MHI4NdaOZjKA0NaO5meVoakJbR5/d2sukHAbyZkA6irA7R7Fq/02MMGvqGlCT6ouvvvwWgTmdNuUANDIBkvxu4/sV3yKpccJWFuZc3F2Nc4e242FiNUfHZ0YSlwJs68ilBAskEm3es8FTDFVi19rVeBrxAUpXfZDNenSUxGDL+g2IKu11qipxh+ttKkNcXBw+dfRDY9BjuLMetV1j5A6PsZZinD+6E9ludvglwx0oLshDZmY6CqtaoNHpoJDJ4DAAyeRhNBig1+kth94AraQPz649Q92AFAb9tM1gIC41MzvoWrkAhamJyMwrwciEAlrS0a+uxihZ49tsQtbzW7hwOQgCtguWo8Nv1k6gMCsJOYVFKCnMR/i7AERnVTAuONyiGpDz5iLW/HIa1b2W5Qa5487eMpu2NfvcFw4Cmw4G9QRivc/gL//5N/inNCwcsu4ysevwA2Qluk/Z77Bj1xEUNw2AzCkrTniJDZt3IbOme3rSuF6B6rx0+HnfxN4zF1A7xD4Bno8Om67W+OR399Qv6/AgpABaE6AcbcD+Dctx1S8F8snf4bq0l9h78hqqWvsw0N2GSL+nCIrKgohZfWtphN+q5fx96vDkyHL879dbkPaxzaX5TbbcSHtY8uHn8Fcj4tA+aPt+0cajXV4uvGneJrQVhGHl96sQUTrg1Ad9orsYu1avwJPYCiauXtKPpyc2Yv3BB+iSkFFNS50ZtQrUfshETLA3Vm/eifRGi58hsWvkfbiydTP8Ej5AY5pZx7KxVhxnJu2SVXpm2q15zOZTOVSL83s24czTBOjMZhg1QgTf2o/vNh9Fo1A3VQ7GJaGyAJHB3tizcw18i/umbNb8RxtLcXbXL3hf0A4TS3ms8dz9nK6judHAXT6uxqfNezZ4Bp0CL09tws5TT9EzrpxRd2xlMihFiHl+Hmu2nEIXs7Qnj/4mPXo+5eLKicO4cOsxsj+UoSA9Flev3UJu4xCTH9PhP7ITWU2W72SVKqVMgnGhEEKbg3wXSSTMCKNysA73bt9l5piMtJfi5sXbSMovQH5eNVQutDHiw//8hhcaXPDhN2nEiH91GwcOnsKbsASUV5Tj/et7uP86CmNyy9uArBe3cP5SIMa0Fi0MWhVEttzHx5myjI+LIFdb5vPIhnuQlZyByo+FSIyJxJPHz/ChrhcGJ/wH6tKx5YeVCEivhs5JXLb6cydsNm3LnXwWelw2HfQKAUJuHcN//5+/431mm0vXzoIsp4MPv0GnRvTTc1i/7RQ+dY5BLujEzZN78SAgielIW7Qwob+1GonZJWitSceB42fxsW36N8zVcrLpak07UJWKNV9/C7+kTzBopMh4exc7jlxD66j1PqVByL1TOHX7NYpKS5EW9Ra3HrxGp8A6L2fJh9+q5fx9mvDq7HL8zz9XIrawFlxzLbj4kPawNMJv+wg0x+ekImj+0cajyY0Py8qb/NiHPzyEHzYdQ5ez/WvMetQk+mL5X5fBL6WK8e0tTiR+7weQ3zq9UgZxZehrKkFYdDK6movx89qfEVnQM/X6S69Rwv/GblzyTcSEZvqNAlm1oKulGSVZcdi1/hvsvXAfBSVV6OkX2Y2485XLJZvZgNaieGz6djmu+WZgQjKB2two7Pt1H6KL2uwgxvvqEBoajtamKpw9tAOPY+umysFENBvQVJqA/b+eRWnnqMtrtNtlwvHFWkcc5tkFm83Qa1SYkMigmzFkPFvoub3GSPtSKaSQylXTo3EclIkrWkn0I3y/Zidy6nrt53OwpjFD1N+I83s24MCjKKi0/EsRykWDiHn/HqWNXWgoisdvO7Ziz+GLSCxqhGbSNUDQWoILR3dNjfCTlZ266iuQlZWJrOysqSMzKxOFpRUYl4gQ/fA8zj9LBLNejUEA398OYNfpB2gWuDbKSDr8Xte8UD/MflFPtS2zHg0fEhEYk4PBvlYEPrqAbVu34bp3KHrF02/lsr1v243wy8d7kJ+ThcwsC3/CPSs7ExlZeWjuG4fZbEBvWwUyPtZCoTVCKxtFiNdjBKdUOp1PoRD14MLeNTh05y1GZOxuFKxV50HglA62acnbTI0aKpXlwcXWRPOctGOlTIyBvh50tHdgaEwMd6eauMJHKxejq7MNzU0NqK+rR1NTC9o6e6HQTLdtNh200hH4XziIP/6/rxHrwpsuV7h8ljgOI/xq2TAu7l6N3UeuIOB9GIKDApFeVAPFpAsZWcVHNtaDcD9vvA4KQ5j/U+zYdgT5lb1u/w6x6Uo0MJs0SPO/ghVr9+JVUBBCgoMRHJHI7B1i1Ug/0YULv+1ARpPFV9woH8TpHcvhm2F927I0wm/Vaj4/A69vwX//YxnCcsrs5ia6woG0h6UOvytKUYrDdQFywpPOhXQczY11qK2tRu+Y/eYjbuNxZjS/BitvxWgrru5Zg83nAjH9887OxagaQ9iTs1i1bhe834YhOTkRoWHhKPzUxaxGYk1FRobCve/DNzgWWanh2LNiNXxT66ZulmTjnJL4lzh+yQudY9MdEq1ciIr8HMTHxSLsfQjCwiMRF5eEqvoBG/9Hay6efxq1UmQG38WqVdvw8FUAklJSEP7+PbLLmqY6aQTdpJEg5a03vN6EITs9Boe2/YSbLwvs5h2QNxk5YU+ZUZghscpzUiwprXXEYpp1EJkDUZqVgNSCakjU9p0q4jdOlkf19I82b0c80n76mssRG5+CrjEpM/LIxZV0qjrLE7Bl+Qq8TiyHxnZLZ7ZEZiO6q9Ow/Ycf4BVf7vRhyKjXQi5XwmAyMbtDS8VCTEiVMBgtI+EmoxGDDfk4/dt2pH7qs/jjs+VrG6YT4s2Di3gQWwWdXgvxUDPu7diJm2+yIVWqXdrwxaASINI/Cl1C9jY5pSnpdCrkUOsMjGuQVqUEGaUnK9+QP6Kf0ahD2pOrOH3OF6bwbj0AACAASURBVINKPfMWy5auJZ79Qx7Z+KsyOwLxuaXMiH9vfQmeP32Bjy3DME4/4zvCMN+1SjECrh3Axp2X0Ngvtn/AZk3heeCUDgTCZMDYQCfqakoRGRSIuPRq6HmWYvU8V0tKyXAnMlISkZmZhcyEcNy9/wKfBu1/X2abB0kvHelGbkYKHl89guXfrMFdL3+kZBZiTDq95rGdDpOZqsUD8D7zK/7wl2+RVDtEg8rnwXDo8I+35mLzmvWIKW6BXC6DTKGeGgggAwSi4R6kx4UjJaccApEYbdVZOLptJ0KSP1pWhHKjFGy6kuQ6SR/uHdmOY89ioNaoIZXKodOTuTrM0wA0Sinq80Pw0+ZjqGjvx/BgHyqyo3Hy1BVUdFt3eV3q8LtRFdSiht7bjz9+8Q3eppXC5pnZJXzSHpY6/C5JRScS1wXIiW42QjrWi+TwV9i9dT1eJNfYRXUbzy715/ti5T3cmINfVn+JMwHZTsiYIRlowvXftuHow2iMCYbR29cPscwyymrFM+uVqEiNYNby7urpRU9nHW5vXYVboflTHWXSiRjvqsGdixeRXNo+wyfZiuWEkMdmpXgQLy7+il2XXqF3eAi9vX0Yn5DbjxYb1WgoTEFUWhE6STm6G+B19Fecvh8BpU3fRjrejWfXz+F9RhU0C3ykfFowE5rzonDjsT86RmXMjx1Zh10pl2Cwtw0Z0UFILJp+QJtO59oZ7fqbiWeGVilCZpQf7r2JgZKvB2k2Y7SrAge2fIdjXrGQqKdHNdlKYzZqUZn0Csu+24zY0g77NsGWgCeMtHPRQDOigl/h/KkjeBEYgaq2IecPr2YD+uqL8DY0GrkFH1BWXoHU0AD4+IWhuVc41TnhyRpmkx7icTE0HB3WmZqyo5nUE6guzcGLG5dw/vwdROWUQShXz+iEO+Jp5WPIivfHu8gE5OTmITsrG5X1nVMPEuy5WULJQ3R20FX8uG4bMj/1uvSAw4fHZ7PjbdSioSwLcVFvsW/TBlx+mgDtHE0cNinHEf8uCLnV7cyqRWblIM4f3onXSQ0ztOXj74qNjFir5BNI8b+F7/6+HmGZlZCRNedt3nbbnlsxVcJuPD75C/7jT18hrrzHGrz4Pskb54ZchIbGoa2tAe8en8WPqzbDPzoPQqn9ynImvQr1JTkIex+O8oZe6Iw6NJVlwd/bG9FJ+RCQ+Db3f2dizNTVDJVMiPy419i+bhVO3XqN1t7xqcEwBs9kgKC/FRnxwXjm+w6Z2bnIy8lCalo2GtqHMH1JaxD88Ak+ti3ttOusHmja313bgf/861d4GVsMd/d1JO1hyYd/Dnz1ibBsB59PHVt8EmYyGjDeWYqDq5fhZnipHa4neFz5zGe4hbcBjTkhWPPlV/BNr7crlyMXnUqC0vQQ/LRyBW4GJEMkIze+aY0JnlzQi5RwHxw/eBLJZe3QaqT4VBSHC79tx6Hzd1FU08q8zifpjDolKjNCcPfZOwxKrD6Lk6OiZLTUBpvmOelMNH5MwZ51y3HqQTAGBDPXGVdJxpAXG4gzJ0/ibTwZZVWjvSoTt47uw96j55Fa2AiVwQyzyYji+Ne4/yIcgxLX9yGYKo9Rj67aUpTWdbP6As5V29KONuD8sSNIqBqYWideNdGLt/6v8Ojedaz57gtc8k9hlomb4upGfdDmzYWnGmnEtRNHEFPRz9teJCMduLLvJ/x0zBtjEv61k8mbj8SXJ/HNpl9R3D7kdE4Lnz6EN3FfEwrHIRAIMD4uhHzSTYQvHWMz6iGTTGBCIoVSpYFWo4RYJIZaOz1PxikGT51xaToD06iHXDqBcYGAKYNwQga9YXrVIGt8RzzxUAcyE7LRJxBjQjwBuUJleQvCw8mKZTZoUZnxBt+sXAf/7HqnPv9T6VzBdohjz9sErVqJse46XN33M648TYBGR0dve44mtHxIxMvwPKi0ehgMBgg7inBg525kNRF3qOn76mzPVZJR5CWFw/vVGzy4eBhf/+8PuPbACy+eeSE+p4LZiZrkYa+DJX/S4X948mf8xx//hdiyLqq8Zlsu99KbMDHSidqaBghGh9BQV4vq6hq0tPVBqZ6er2XBNEGtVEChUEKj1cNkNkGjUkKhUECpVPHuI8HGiU1XrVqBrtYG1FRX49OnJoyJrT750/Wu12mgJHkqCBfLodboHOpJj+rCYvROzk1iy4uN0+cIo8mN9McUcinkKg2MZI8PtYJZSnW+yvXu8s/4z//5Cj4xpMM/XWeu5E90cGGE3wy9Vg25QsmsoUwKrFLIIZUpmNdArjy9EDI0/xzxCCe1WsNcEDPzIZ1mPdN4yUVE0uo0SsikUijVZJ102tzIGu1qyGVSi0Y2I6/kFbxUKoNcoYBWb4BOo4KU8FBpZryqJpwV5JWfXAmyKcyp9StwM6LMrniOOkwZyeRUlQIK0ijNAPPqXya1/PDxjUhOAcztCcPbpERB6FN89Zf1SKzs481wou8TQt/6IyAgEAF+/qhos19ikOCRkSTy40UatZGUkXTsDQam40jaB3ETsa1psuxbWTpZE7nCbhSPU1Nehq4ZVePdiA3xg79/APz93qKg0n5FHoJCRmZJOYzMkog25TCamIc/YiMcx1uK4fPmPXpF06/GXWNhiUVWdylJfIfAlAqLr7ZDYkcdSBtSyOVMuyWvnknblJMfIZMTHwlbXKMK+cEPsef4U4hsvHYsrhtGKHorcOCnb3HRP9n5SLQtrs25I28bE3Nqua7k0Oj0zEMTuZfJFMQNhr0c3HhGVITcxI5zzzHMrLTkmJPlu0o4iJcndmPtpnNod7IWPnEn8Tu/Cet2HUdj/7hde2VH5w7l5s2dhs9CE48mFuFsi0fa0mh3NfJr2xkfV1sbX/mmbGYD2ssSsemrlbj7Nt+pW9VUOg9O2LhJh1tx/cAvuPos0W6JRg/gWZOY1CLEBL1BdlER3r56jhfPvXDu2CHmPmB7fySrI2lUcgySjcZ6etDb08ty9DC24XGJfVs1m6EQD8L3+mGs3X4Mn3rHUZ7og1X/2oac8k4MNuZjy48rcNUnFkKFpdPrSFYt7seL07vxn8Slp2YRu/Q4FszJd7Y24SQJp5kmFsmED4/Pxklwngy0uJn0SpRnhOPw/j04eeUB0rOzEBDwDl0THMuHzUH5Qu/tw//736/hn/xxblx6tAohqotzEP4+BKl55aitqUBBYR7iY8IRn/kRQpllS2i+snEJbjIYoFZanmCtT5Jcn8wP9OQdyRFvrLMcIcEx6BomPrVGiEb6UFNZie4hIdPR62+tQkJUKMITMlBX/wnFhXnITIlDWEQMmgboLr8m7G9CclwUElPSkJ6aiKSMQvQLpMyrcI2oFxHBAXjm9RRxaTkoKixASnwU3oZE4FPnyOQrfBMUoiEU56YjPikFBYXFyE8Nx84V/3K5wy8b60NuZiLeB4ejuOITqspKmEluERERKKhocenVNnt9mmHQa+2e/Lnqi4xSkDWG2f6Y+tNKkPDmBv6ybCtym0fYok2HkYcy668R+bSeT8ZwbA/TCfnPjDoNZBKFnZuCp1j8OVmJ2nBnKYdLGEwk4lYih0LtWWefQJAO/8fk9whOq3Le4Tfp0FiWjeiIcCQm56C0tBQ52ZlIiIvHx08ddnMP+MqgE/fhxfnduBBUwBpt7jv8evTUlSA2OhKRCRn4WPYR+bk5SEqIR35ZAxQsmzLxtQdJWzo2rtmL4nYha3lIoFY6ine39uHHDTtRM2j1f2WPTpZrfbhzA37aexMdww6dKPYknKF8vDkT8Rho4tHEIpRt8ci50aCDjjwYO9h4imdjMqK3Ohs7/7UC554nz0mn25qZLW9rmPMOvxk6tZJZpWh8fBxcx4RM6XibZLKQ9tcjICAaHV0tSI2PRnx8LG6fPQSv2CL7Zf7MJshEoygr/oDcnDzk5bIfxFbT5DCh1GzGUGcFdq7+CjvOPmc2fiqLe4GV/9qKzPJOKOQDOLV5OdbsPovWQbFd/Vl1cKfDT36DLG+xuPXg0mmhhdMsB00sohMfHp/tc2gsFouZgTPSnmyvMzJYJRWJOK8bwlUompixARrB0UlHkJueieq6eqRHB+Lm7QfIr7G8fWIekNUqqFh+Q6xtmsbn3Hb4TXo0V2QjJCYJr+8cxIoNvyI4IRftPf0oTfPHTzv2IrO63ensflvBpwtthlIwxKy/HBkRiajIKO4jOhqlTT2wujQ64uW+uYyN206hsnUQ7bWFCHj1CLu2bsObpDLIlUJE+XkhNMgH2zauwrmHAahr60ZXXT4uH9yEKz7Z9je6aYIenZVG3sPaTbsQkVuJrrZ63DmxG9d8YjAm10GvkqAoPRTbvv8a2w5fxcdPLWirK8bVoztx9HEoZGods0Z71KvrOHbtKaoa2jE0NIiSlCCs/PqvuOXCCL9ZJ0V+fBTictJwcvta/HzkMjKKqpit4MO9LmDvsdtoHfF0100TBL0NSI6LBl+dEVt0QiqaB9k3HCL1R/xIw5+dwZ9W7kJxF39HyFlFOLYHZ/H57DSx+PL53Damw58S6lKHXzFQiWfP/RAZ/Bxb12zB03ep6OzqQkaYFw6evIZ65kfbeYnEPbU4t30T3mQ3sUae6w6/eqwJL577ISXhLbas/wnXvcPR3N6NkvT3OHzkDAoaB+0e/ghJvvagm2jGwRXfI6KojbVzRdKTSeTRzw7h2zWb8MHJ7ojKiW5c3LgaP//2GL2jZHMbz//4eHuCShOPJhYpCx8en41dBxOGmkpwYNW/cPDuO8athj3e7EPZuDnv8Jsg7OtEbno6UpNTkZaSNuNISU5FSc3Mt4eAAa1lOQhL/mCzSo4RuQHn8dPZF5A7js9YByVc+bSVY3KE/83dE1j/y1HmjWxZvDdW/msbsso70VmVjq1rV+KWbxxECvbViNzp8Le0tCA9PR1ZZNWmRX5kZmZSKwNNLKIrHx6fbb7rhHApKiqCUmlZVcz2OjPq5KjOz0EKy3VjuZZSkJFbYjep3Nq0iaeATm9AT0MFkhJSUd/WzwwIaNVyNFYVwv+ZD1JL7Vfas6al9TnrDj8Rg/MwqNFaX4GGji54n16HbSe80C1SMW4wzTn+2LBlH3Jr+6Z8cW1xjGoZKrPikVfZDM3kduq2dnJu1OsYX1GRSMT4iBI/UdZDLGZGM4n7DUln55OlF+Pu4Y3YtP8M3oeFIy69EEODA6irqcagQAKttB+FOQXIj/HFmnU/I7W6h+Er7avFhT1rcfltkU35jegqy8XDG1dx9eYD+L7xxxu/13gdGIaGHgFrOR3LJBnrQXX1J/T0D6Czow1eZ7dh3a+n0TgwwfAebv+IAxvX4axPEqRqPbSyEQRc/Q0rDt6DUKZAV2Uyfv5+JZ7Hl0I7qZugvQgbV/7LJR9+vWIcpaWV6Gsrw871P+C6XyrEGgPj1hP99AR2HL2PLoHVb90E4WALQv0e48Kly/D2DUCAvy+ee71EdlU79CzrUOu1KkgmxOCrM8Y2IWFW33DUx1p/RvkYQh8dxx/X7EFZz4RNHcxsj8PDw6irq0NDQwPrUV9fzxrOFZ8vnCYWXz5zYSPcyet3vd7iumarPZkbYFu2urpPCH31CPd9Y1AzqSuxNzZ3QCxX211jgvYy5Fc0I+XNVWzefQY1XaPMG4Ka9DdYu3w94qv7GLe5wfYq+D67iQtXb8LPPwBv/F7B+2UASpstfu5DzUXYtfEXRJX3sNa3vKcc+zctw4U3STw+/EYM9zajrp67LdTXN6Cts88yIdHm/ibubUBOeTOKY55g09Z9yK7pgcFoQFdVMrZ8txZvUqqZkWHhYCPe+9zFxYvX8NLPH/5vfOH90hfZla12frRG5SBObVmGJ8lVnPcGg3Iccd5H8M2qDchrFrCW21pPCnE7zqxfiR2HnqFvbKZvrTWeWq1Ga2urXX06tifbuna0efKdJh5NLFIWPjxHG/ne1zdzPwurtsQlaKS1HIc2/B17b/pBPekGOm2feX/y1Gb3OzbZTqVDLRaXnqeJnD78ZPM+uUwGGdchlUHJMl/DqJEiNykGWRUt0E+u5GRUjeDluX04+iSK2RNkuiwmiMcHkJuVjsSkJCRxHGRn5ZLq9pnzTUxGyMYHkBkbipev/PHossWH/+oD4kbkjdisEozLNEw6Nh1Uoj48P70b//HnZUisGeS9bjQaDaOFXC7HYj9IndIqA00swokPj89Gqzzu4JDOvrVdWT+Ztm0yMi7phC/nIbe4r09fC5Zr3mjUoq06H2ExWegbk0CtEKK1tQtqgx7jQ814fOQkXidU8LZVO0wyB0CjZlxl1VrLHAk7rja/XdZ07+/txR/+9jXeJJVAPTc+/IB2rAa7v/8Oj2NLLbO0DSq8vXEAe88/Q5dQPTWRyvoUQ26a3dXp2L/lB9x7nzm1c5vVbvkkHX4tpGKR3QYwtpvB2J7LiJ87GWlwGNGR91di+/L/weotu3H2yh289n+HD9WtzPrLzJR2sua3fAzBD07glxMPMTChgNmkQ11uKDZv2IncljFmh0irL79GIYT31X24GZQJmdYIjVyIl+f24sy9dxAo+FfZAFlWr6kYLx/fwZPnbxCfkoHbx3/C2t3HUN8nYiZ5jHR8xIHNG3E1JA8KnQk6+SiCrh3EdwfvYVwqRWXSS6z48xqE5TYxE8ZIeUWdJfhp1dcujfBbFDKhIz8Eq77fiOQai7uMVtqD01tX487bbEyodAwXJq5Jg/zg+9h1+D7GGA8RAyqj7+G7TafQLHBcLNMy/0Esst+0x7aeps5FYmYd8ckqs9Ca/E8ar1EhQNij4/jT2l9R3iuxszt+aW9vZ0YXcnJywHZkZ2ezhrPFdRZGE8tZXnNhr62thVY709VHMtJpGRFMS0MaOVJS8Pz2JVy8+xJJ1rDUNGTmFGNQJJ8xt0Uh7MPdQ9tx4kEoBDId9GoJ4rwvYNkP+1DcbtkDwKwTIfT+eRy9E8LsYmzWShD77ATW7HsEGcwYbPqAnRt2IrqCfc6GSyP8ZgNaKidHaKy8bT5TU1ORlpqGDx9rMaGYqYNeLcPri79iz7ln6BZpYNJrUBz5EF+vICvjkFWbzDAbZEgNeoD9p15AqNYBRg3yQ27jh52n0W+zNrtRNYSTW7/B3YQKzjkHRpUQ8T5HsWz1RuS1WNa0dmzf1u+KiXac2bAKOw97oZ9nzXviwlBcXMzb5mm3Y5p4NLHINcSHx2YjDwnkHsT+Z8ZoWwUOb/wH9t70nb8RfrMJCqkYXfUlOLvzJ5y6+Q6DI2OQk/ZH5c8MhbAfL+/dRFB8IcbFEoiFw/iQHIxzV+6isd9hF2+zmZmQONzfz/ju9/X0ge3o7e7D2LiU+w2XRoXxsUHE+lzBN/+7GkEJHzAsENq5SrHVhUrUi2enduI//vsbJFT1U1GAC4SsEqaeg/l8XPnxhbNpwRefz0YTi+TDh8dn4+M4H7bZcyNudFJ8zIzA+UvXERaXioKCfLz3ewafsGxmsMegGoPfmQvwS6xkikQW1dBoVFCQSdh2h5KZB0d+dzprSxAZHYeUxBj4h4SjpqEJTc38K6mF3NqNP/z1a/gmkA6/e+oRHVyatNtXEIDlK7chq36YubAVA9X4deNKPI8pRv2nWrQPWUZordnrJcOI9n+OE4e344JfAsQqx3eFJKYZKtEoyvOykZKcgtSUVO4jNQ017f2Y3PHZpuGZ0ZLvj2XfbEBCVS8mBINI872JtZsPoLB9HEM9HRgRqzAx2IJrv23H5dfJkKoN0MrG8O7eCey58AK9vW0oqayHcnJyrWy0AWd37cL7rCpMyKTobqrCk1tXEJpaBuX0mlTWotp9mpXDuLJ/JX45eRv1AxOMD1ns4wNYu/sIcgsr0NYnwkhHKdPhv8LS4RdKFWgticGmZT/AJ7ECusmJhML2D9i08ivXO/wmNVJ9TmLVruvomCCtwoj23ECs3bgHGaWNqCqvgUBu6cwbFMN4fWM/Tj5PYdYFFgz3IO71HVx/EQWhyvEBhyz114rcjFTeOiP1mZaZh44R9h8C0vCYDv+Tk/jT5Ai/nZBufpn9BT2dIU2sadSFd8a49KSGIiTdBR9+ACOt+fh14zrLcmBGQDrcjKsHfsLB2wEYlViWl1ONNOHG6QN4FFkMhUyC4a56BDy8isfBWcwDgKirGme2bEPIhw5WQVzq8LOmnA50Vn9KYRuObV6BG75pkOnN0MhG8Oz0z9h5+j7aRy3tVSPux+ubR3HVLxkisQRjA+2IfPkAN15GQWqzGZZe1oWjq79FQE6D/dJ203RARvhjnh9mOvwFrc46/J04v2E1dhzyYkb4bWDcPnWmg7uANPFoYpFy8OHx2dg1MGOkpRSH1n6J/bcD5q/Db9KjqTIfEe/f4dalc7h2xwuxiSmo6xljp+luKHlz0V2HN6+9ERGfgtw84pOfjdTMLLT0jU0PALmL6yQ+We8/MzUJb17cx8ljZ/EyIASJaXkYkTLbujGp2epIIWhjBsv+739+gegPzU5ymZ1ZMtiGyNBYlJRXoqqqasZRWVkJcrDZqqostsrKmelIfCZtRSWq+Ow22I75WNNz4ldwc3Oalq9clVWoZLCny8XKbVIvVptDelv9rNxsw6bPKyfz5tCc8CaastQVCSO27kHB1CAMW/tyq8WYTRCTfSvS0vCpoRFxb1/iyuVreBUShz6hnLn/OHb4yUqA3R3NqKquRFV11dRRWV2DQYEMgo4q3Lx8E0UtQ9DLR/H2+W2cvX4fHxr5J6gHntyAf//zV3j6vghKLdegBXvpiA7OO/xGOeKfnsG6X6+gS2J5pOgtDsHyZbsRn1uAd+/C7CfeGFSoyIxC9sdPCH99Bccev8e4nH2UgkyaFAvGMDI8gpERvmMUEjlxJbIUZKoCTUrE39uHH/ffQ7/M0jkd+BCOFT+uR2h2CWJCI9AyJEZfTSp2bdmK0LwGaE2AZLAFl47swO2gBKSH+yEmr37qYaKvLBS/bN+FoOh05OVmwPfpfbxNKICQZbTQUVazcgBnfvkWO88+RPuoDKK+elzevxarN/+Cu3cfIiqnDq3lqdi+6gcc84rCsFAK4UALnpz8BV/uuIDWvjFMCHrw5tZR/Hr2PpqHxFBJBcgJe4yv/vZHHH0Sh3GZemp97ikdHIgYFP2488v3OPQ8FWpm8REVYp+ewfb9N5GRnYagyFQIFZYb7kRPDc79ugO3/SNRmJ+N0IBXePQiGL0Cts66GTqVHILREf46Gx7B6KiAdRIkoUp4m9ViRL28hP9asR2FbdM75ToUxaWvXDq4lNghEk0sB+gF9dWtSbsAqhNf4ru/fY0HwbmYkImRGHAHR87cQknzALPRE3mo7K3JxpG9e+EdGo/8nAwEvnqBl8GJGBZbNjhTCdrx4Ohm3I6psNOCbGY1IRSiqzIDP6/6Bw7eC8TQmAgyxrXI/ZuaHbjDl6HaRHz3P1/gwrMYjEslKIp7jUNHziCronNyopYJY13VOHdkH+77hiIvNwdhAd7w8gtDh8PAhnKgGFu+24bMOu6bNBlceH//AFas24KyXv4Ov0oyiFtb1mHrvnvoGiHvRDz/o92OaeLRxCIK8eHx2djVNaG/Nh97vlmGk4+j56/DTybJToxjaGgYIyOjzO/h8MgIpMrpjjE7X9dCzUYDmkvTEJNaAolCjrHREQiEE9DoLZObXUNxP5Zeo8DY6DBIWUZHRzEyPIzhUYHdRH+2OlIK2nH3+Bb8+x/+gZjiVvczdiNFXUYEvP2jUVlTi5rqmhlHFem4llcxy2nOtFejkjwokKOyekacqspKVJRVorpqJi7Bqq6qZtJZccl36zn5JHmT9FWT6atJJ7eyCrXVFq6VZaTzy86NYNmmtcVlsMt5ylVVjcqyClTa8JnBjTzETOo1basGeTghPEn66XD78ldVVDCaOnKyfLekJXqy2asZTSuYfGomdbCNR7TuGRyn2OG3rFSpVFn6XsRnXyAYh5IsVTp5kyabDb45cxFvkiwj/GQ1OBlZUnjGxHohCE5tfhRO3PWFUK6BQSnAu4fXcOXZe0icvNELOvsT/v0vX8ErvAhKnXu/EOQ6c74Ov3YCSSE+CEgqn/LpJqvNBD56BJ/AcFS29tusQ2pCX00ubt16htyiD/C6fgp7z7zCqNh+3XSS8WyOKT8ngxiJPo8Rkl03tZY4eVoKe/MYj18HouhTO+NH3leXw6wF3DZsWfdcr5aiKD0Cj5j1gMuZUX8LHz0yfK7i7K0gDMvJ+rNy1KT6Yu++y6jrE7nA2Yi+pjK8JR0d30AkpmWivKwQrx/dhk9wPFqbq+D18BZOHj+G05duIjy1EMnRAbhw5iSOnDwLr8BYDEk0kI73Iz8lGm/eBCI8LglZKdG4fO4Ejp27joi8Gsi0lrXip3Rw0FMn7oL33fsoaCGuFkRrE4abS/Dkzj0ExWSga5jMJyDhRrQURWDfmRtoGRhjyttbm4ljB3czbhezqSO+tAxvgxzZIQ/x5RcbkVIz4IK23G2GSwc+Dlw2mlhceSyEcKNew6zS8y6tEhqH9kP42eugR/CtnViz+TBeBQbhjb8/YlJy0D08wSzVy8Q3qFCS8Aonb3ujY1DELNtZlx+FA0dPoKh1sh3qJhD78jL2XQ+D2mZ+iFY5hoS3fnh87zZOHT+CUxeu4ekzP2QXN0Otd29fBHvejm3GhFy/0/hm5R54+wfCPyAAYVHJaO0fZ5ZfZOqFLFTwIQqnL1xFTccw40/bXpmNs0cPIfpjJ/Owbak/EzqyX2DdbzfRIeTeB0EtHobvuT1Yvek4mgQS3naukYvhc2Ij1u86g+YB0Uy/aJZ6snBxLKdj/c20c6XjCufX1T18mliELx8en429rAZ0VqRiyz9/wA2/LMb1hD2ee2Vmw3Cfm+d5GnQaZIW9Rm5NL28bZOM512FsOmgkQ3h9bj/+wLj0zO73gZe/bgKhr7xQ3StldqI2Go0zPw1GGMnBZjMaoZgQ4GNxEbqHccUUtQAAIABJREFUycowemg1GqhUKmaCJ0lj4EnriMksyWybj8E+vVIqRGlhETqHRBZcHm4EizdvnrQML7I8tA2XGdwcbBrFBGoritDQNcSUndFBqYRWp7PDmcLm08Uhb1se5FyjlKGmsngyr5l1Y+nfWK4XtvbF2yac3Gcd8XRqOWpL0nBx915cuvsalW39U31lrnzGe+vxxj8QWQXF+FhcAP8nD3D7kR8auvjneYXc3oM/ffEN3mWUMju3c+GzhRPezkf4XXxSJp1HyVAb4iMikVlQhPLSDwh8eBV7D9xBt4Dult2kMHPxZ5QP4Nax7XieZnmFaNAq8SH8AXYcuIKG/tkv30mb92zxjFoJkl7dw6X7wZCqdDAZdGgvScD+XTsRO4d+kxbeOlQnB+CHvy7Du9zZvbKdrQ62bYkmli3uQjsndd1SUYC8qg7G3caRn60ORlkHjqxZhnN+6VA6enlNJlRLxxB0+zyeBKZBrjUym5tVpARh94ETKG6fdksYb8zBkf3HUNwtn9UItiNf63db3tYw66dJ2Y/re9Zj/513EKrZ7yEGrQJJfndx+f57CORaGA1aNH5IwJF9BxFX1jX1dk2vGID3xYPwjqti1c+SpxnS0R7c2r8NG359gEGxZdUIKx/HTzK/IPLhHizbshdlncNTo0eO8Vz5zqeDK+kd49DEo4lFePLh8dkcy8h8N+rwKTcY365YgxdJVcwDLWs8CoFuc5tFnga9ErXFpRiR0HljMAsqM5Ky6aCTjeHtlSP4r//7JcJz22ekoRUw3lqIF77xdjuYu4dtQEV6KBKya6A2AgrRIDKTIvHgwR186hpx+x7HpoUdH5MeHXVFeB+XCoFs5q7TtnGdYtlGduGcD48MKnaUZyEiIRcitQFGpRjFeSl4fOMB8mo6nK7k6EL2DlFMGOupxfuwaPSO8a86yMfbAdSlr454ZC7q2HA/evv70dfTg8FxCTi2dbHDV5Pd5QcGIZLIodOpIRgbg9LGZdQuMvPFDN8LP+LPX36HiLyKKa+UmfHYQwhvSh1+MwZbKhHq5wXvoBSMy7VQiHsR8+oO9uw+gricSrC68bPzchrqKLjTBM4imE2QjA+hJCMcv/20GT6RGSgpKUZuejy8nzxEWplll1ZnMM7stHnPBo/4mDUUJ+Pa2aM4e98P+R+KUJCbicBXPvCPyMaE1s0ZIc4Kb2O38DajpzIRP634ArcjSmys7p/ORgfH3GhiOWIvpO+knMzGZGTnUhZijA5mMySCPuTH+mDVihW4+iIE7UMC2Owlx6Q0KEX4mB2LU0cP4LZPKAqLipGXnQLfFz6ISi+H3LYtGeRID7yPB36JLHNEWIi4GcRVf0rxMD6mBGLbmhU4edMX9d0jM26YRq0MdcXZuHluL84/8EVeYTEK87MQ4PsCb+PyIJJrGa3MBjXqciJw8cYz9E26ObLSJBuk9dXj6C8rsefue4iU7K6N1rQmvQoF4ffxzx9/RnqtZTUxq83dTy4d3MWxxqeJRxOL8OPD47NZy2b7adKr8SHqAZav2YT48o6phRNs49A6d5cbrXwXGg6bDnqlEOEPT+C///0LvMuY3YAQZ3mNahREByDuo/2cIr1Oy2wqqtNqodGybAxG3ioZLZs8GuT9eHb7PhpHVUw2WqUElR9ScfH0YZS1DrDeW+35kDdUhslRcAPUKhX0jjdYpo2TDRktu0yrRb0IfuOLsvZhqm3fnhdg0Oug1euh12uh1kxvUmo06EFWRiKrhanVKsZmNCrx3usecj71M/OZTDoVWqqKcfPQWSSVNLm09DmZ6KojbwMMBmb1GrIEpuW3ibjUaKBh8lNDpVZBqzPApJ1AcogfEotbHKnbfWdrX3YR3PxCG8/17A3wPrUcf/3XOsQXNUDP9sPNA0Z4U+vwi4a6Uf/pE5rb+hg3Go1chPbGT6iurkYzWa+UfSNLHnrcJuqCm01QykToaG1CdWU1GptaQNb2bW5pQf+IkFnCjJuN6xbavGeDZzJoMNDZhNraGnyqb2TKS8rc2TMIxeSW366XzL2YVt6S/mqc3Pw9dtyOdOmGwJWLFY/L7k64MywykkFeKbp5rblDYUHEZXQwmyAV9KOsuABZmVko+PARPSPCGaM1Ro3csl17TTXqGpom21IregZGLUuzOpRIKexFVMArJOTXWlbTcrDP5itX/SnFo6guLUR2Zhby8j+io09gWXHMJjPS4e7vaEFtbRU+1TdY7gHNzejqG2LmozB1btJjqKUUr1+9woe63hk7ZNvAMa50A/XZ2LHqG9wPz3c+6d+kR0txNDZ8uxL+abWzuu+w6kB2cdbroCc/pMwDn+WHW+9Ce2bFsy2sG+fTWJaN/HR6y6IOpCPB7JhudYx1EXMab2YCPtvM2AB5yxL98DjWbT2Kqq6xWb1lYcO3DXOXm23a39M5mw4mrRRJb87jr3/8C14l1sxJcdXCLgT4BqBjbNoDQS8fRkZSFAJDI5lVsGJiEtDpsCeGViFBQVYKGrqGIWrKwdmr/piY6n2ZMdJVhZsXj6O8bdAyQEB2gWc6zqTzbH8YjXr0NVUjIugd4pIymGVQ0/PKMKGyH3Abba9GTFI21GRDOZ0E8e+8EZzLt9oU/4OwM0HNWhlKc+LxLuQdMrLzkBwXg8q2USaZbLwHsYGv8MIvDAVF+YiPCkdKRgwuX7yLxlHryn5mSAa68OzUZaRMdvjJbydxC3LUgHwnv6miwXbEh75HZEwScnOykJqWjSHmrage7VWFePn8ESITM5hBmNCQSDT3ClCeGoSHIem8xWFrX7wJnBhp4znJzsasxv393+Hvy35BZlnn1BwFmwi8p4S3cx9+J/5MBMTZ4ejz5Cy+MzttPGf50bLT5k0bj1Y5neFYeRuUowi4uhurtl7CoNZ5O+LCteJx2d0Jd4bVnPseN56+Qde4iuk8jfa0ovBDMXpGJzjXYncn/4USd0oHk+UBx+pDaesbORuuSqkYo6NkeT73fPSd5TnFe8Z9ybNyzMAzGaEUCzA6LmRW0eLjQ0buatJeYfUPW5BU3m7j/8/V1k0Y667B8Z9X4qRPElQ6S8ecLw8u20zeJmaBgOggL9y8/gipWQVIio6E/2tvvHgViq5xy2oTLuPN0JerTDPDLdxMEPU3I+79K9y8ehfx6flIiY9F4BtvvPAJQesI2Z18Zlo2fjPKapOOz8aGpZIO4c6RLdh10Qd9Yo3T3zY2DFfD3OXmKu5ii8eqg16JnIjH+Pt//wH3QvNcbguul92Ezqp8BERkQKayjOKTtIK2emQkhuHChcuo7hxGvN913IwqtWkHJnSUJWDvlj2Iz2lA38dIHH0QaTMnyYThTtLhP4aytgGGt0omRHlhNlLTU5GWkcYc5Dw9Jw/DIinqK/Lx6NYteIVkYqCnA08uXENx/dBUnhqJAGFPL2Dz0euY0OhhNimREf4aj6I/Up6/Mn29KUb7kJeciFvnzyCnoh1NGT44/egt5AYzjJpx+Nw8hifRVVCqFKjPfod1P2/EzguP0S+1Ypgw0d+FpydJh7+RmWOpVkygujTfToe0jFSkpuWgVyBBf2sZAl7cw8WnoRjq70XA3TtIyK5n9mQZbivD9ZOnUdI0AKVCCv9Lx/AsMAmV+eE4+yJqSiu2+mdtXzb3CLY0fGG08fjysrcJcWH1N/hq/W8obrS83bG3W7Vn/yS8KY3w8z5YMJXBH8M9KynkYvyjzZs23nxpOs1bx+w7sHzFWqQ3ijweNZ/Gm30JeLEMMrw4vR1bjt5GVVsPyvLS4fPwEjZv3ovYwuYZLiKzZ/P5EHh1+Hy0nOZMm/ds8IwGDd7f3oON+y8zu047JQ9AKx1G4K3DWPvrDQzLPfe3duRN/EwL4wMRn5KKG79uwYbdF1DVNQ61sBMPjm7EsccJmLlrgYmZHCdVkI3Y2O+5JB/yQ+L0sLlnM2nUYuQmhCOntAj39mzE+p3n8KFxEDq9HF7HVuHg3QiINOS1sBly8TgG+vvRbz0GBjAulk751zuW1VZnPpttPOv5eEcxdq1djicRhZzzVaxxZ/vpLrfZ5rdQ07PqYNahIj0Y3/7tTzjhHe+2+4Kzsho1E8gh7a+ibcabtNJEH9x4EQuJdAQ+lw/AL7txEs4M5XgXEiIicP/cBUSklGCoNhmnbgdDPjm3iazO0tNUjCtnDqGwoXuqjXLzMUM61IoXD28h91MnBlrycerETdR0WeYMksUVmms+IjrwKdYevYYhsuKhQY7k9z54nVLNPFBwYbPqyhWZJbyvOgmXbvmgTyhDsvdl3PBNhMYMKIYbcf7XX5BcN4zx4W4kBL3EU5+nOHPlIdqFljcTxD1H0NWMB0fPIS6vFmoD29Ls9pnqpKMIfnofkbmfMD7UihsXzyG9optxEWrMDcLhc4/R3DeKwc5q3Lt5C3m1vajOeodrb+LtgRy+zVYHBzjq/VlHfM7vyi7s+fYf+OHXM6jun34rxRnfwUB0WOrwO4gyl19/Nw1vliLZ6iDvLcX+dT/iXkgBM+nJE2hbPE/S26bhw9IKmnBw7ZfYtPsUnvsFISmvBhOiMbS1tkOi0Hj8wGKb/0I559NhoXBk40Gb92zw9JJOHN+yEpdeJUCs5pjt7FgIgxoVKb5Yu2YLMpumJzs7RnP23ZG3kUzIb6hGY1kWft20Do9jShkIjbAT9w+txeb9zyFyANWrJ5Ab64/QTO7Jq6qJMVQVfUBeXh7y8/JZj7y8fNR1j8I4+cxAuOlVMrS3tWCgqRB7N5LrP99y/f//9q7Eq6kkX/9X7/SZ96bnzfSb5Ux3j7a9uHTb9j7aTNuurdKuA6LYoqjdoqKiqIiAshoUEBXZBIEACQiyBghbCIQkQNbvnUoM5F5yKzfhl5jMiefck7r1q/p+X31Vwcq9VfWzG5Cy5V2s25yCvhm258GOPmUNUhLjkZB6ESWKByguyEPa+Qt42tIDy+uIsSLqi7diHRYNEonG/FNY9eG3qGwfloytIFE14OxAuQXsIEoq+NbBgZ7qYqz701tYf+AqTP7niwG01omp4S7cunETnUOiB03OeVz59/c4eD4btZUlOJd2HZop9/p8m1mH0ttXkffgCX5L2IELeQ+gH2tH4sFf0Dfj/n6bdEMouXcTiYficbOgAkOvjySWJMeCdbZUYu/OfSh6+AxFt9ORpahdDFY63t+KG9czUVpwHR9+vgtdujnYZrW4mX4OZS3uiOVS2AJdbXMY0mgDWFrtQPmVBOw6koLamkqknUtDa7/7R0hPfQG+2xyPpq4+tNaW4eTJVNSoXyLteAKqu9xHD9vMU6gqy8eJ/Ydx6cY9qIbEf13ErJ2YHGjHv/fuwfX8CjxV3MGVrEKMzbLHEDbcOxOP/adv4GVvD8rz0pF0OguTZhPKsy/gukJ4zPMyZK+HDWJbMPcCXYMBCLKOTduI9/70O2zafhS9U8sfz/iDZbxjE35/KhHaqQcKNR5hU7lQAt42M+4ci8PauGPonvSs/+NWX2YU4C2zBpbBwxp8nou//f5tbDuYjDNnUpCYlIz8Ry2kG9IDYxu60jwdQud15cjUvFeC1/3kOj5Y8ymKX2gWJ7v+W+jErKYJuz//EAfTyzgnAPGRxLzZvcM2h4aidKz5aBPK2t0RuLXqKny9+s+IO1Eo8OV02NDf8ghbNryPzan3BOeme3u2mAzo7+pCh7oDnR2dPq8OdSc042yJjrsm4+Li47Cho/w6Vv19NQqVE64fzNaxZmx69x3EHVt6wj890o3EuC9xNqcCLDjhxNArJO/+FofO5UE/51725M3JOy3Wwdu2LG2ZxOntn2LzkcvQzgrXUC8rS5AREDcCf5EKIaWDvqsK//zkj1j9xTGMut720LSAje2uxke4kVMCnSi4pH1KjV3bdyD/SSPa2tXQjOndT9EdFvQqa1FQ/ADKNiUun45HcmYR9GYTCq6m4kFDj2tNtWXOgIH+fvSza3DYb6Rkh3UB9cXXcPT4r6h9oUR75ysYPCFUrUZUFt/Fw6oGNNeU4NvVcWgYNkDbo8T1GzkYnGDHjEtrItDVMIC0tCxo5c4TF/Q4cyAOF+8+RntbO3o0o+7gbM55VNw8iyO/5biOMjaNvsSxA9txuVyJBsUNZJfWY94OsM3vw5pBtw4Dg674QdJM2Ys8G142liHhUDIqapqgUnVg0nMK0dwYkvbE4V79IOzsiPGWQny5fgeaOjtx+9oNtPbx4/gIdOCSkGekxpPnFdC1FeCd3/8XtiVkQhfEpljGOzbhl6s2QTnqgUKNR9BEWRBi3uOdj/Hd2rW4Ud7mOr9WFohXITGelyngpCSWcwFlF7bjL5/sRpt2GmbTFEov/RvvrduMJs0UOpStGNG7A0wF7DQCK0jqEIFcvSlR8w4Wz24aQ9q+L1wTV70nRLg3UU7a6bCiLjcZqzZsQ5vWGNSbI1+8rbPjyDq5E+u/2o/eyVk4rLN4lHUS/1izCRVq94Y8D63ZqWEU3EzHgfgf8OWRa67N1x6b4NPp3hzI/PGvpVoebk6LAXfPbMd7H+5Er8kBdpBAfd4p/H3VBpS8YOuf3XW0HZXY8vU2KGq7XWHq1TX3cfjwIZQ39yyed72ELkx5fAlzfdw5HRhuzMenn2xAYePQom8fJcmyZHMj8xiZQFI62Gf6cDBuA/5v1SY0jQT3MMhXi62WOZTduQrFM7Vr35WnDDuStzY/Dd/G7UbzIDta0b2/yG6dQ8fzCpw+fQHK/lGY9SO4lnoIPySccUWTnxjqQmHxQ2gmjdwJuMeP96dhrBsnD+1E4m93XRNcu+s8RyeM02MozzmPExn3YbTMY1D1FFvXfIorJTWoePgA9ape19stbyxxWqDrdDeOHbsIjYxVgk67BZ2Npdi8/huUNvcuxihhy5VGuhuQeGA3kq/kQ61uxcOCLKSev4zu0SkYdMMoLS1Bh2aCu9RIzJPdzxvGcfvcYexKSnMds8m0Z8v5FkwzaH2Si82ff4fiWiXam+qQeT4F6Xefoq7mER49b4fZwn/9I9DBl/MA86jx5Ll3QJmfht+/9Tsk3X4Mi+ePo7zKrlKMd2zCH4BgKy1KPVCo8VbaPrn1xbytC7O4l3YI2w+komss8D+aYjy5PHyVk8JyGAeQtOUz7Dh77/UTfRua76dj7fpNyC2tQGZ2ETR+zln35S9S86R0iFS+Hl7UvIPCs1vQXp6JDRvjUD/IPyPaw1v8aZkeQNLWjUi8+jCok4yW8XY6MTXUgUNffYTV63ei4oUSL6pLkXT4AK4r6l0BpjwcnDYzqkqy8PDZCxRlnMTnW1IxNU/3xNvDzTSmxqEv1+C9j39EWVMHmp6VIiF+DzIULBjcIhs8z0nBl99tx61CBRT37yE1KRG3FPUwLLj/o/fgeWp4f/Js3uUW9AOu9dp7T9+FkfBEOW8f4rRcbuJ6/2n3kjo4zLiXsgNvvfM+8p7LOeKSKcOOG7a6AuZZfBxvyUqYpnqR/usFdI0Lf0RY52fR3amGUtmKwXH94pIum3UeA6/UaFaqXZNydgJhZ1sblK1tGNHNwsmCbxn0MJjZJu/Aeseo00DZ0gxVRw8MZncAP9YGFsxL1doMdc8g5i0WTIz0oa1NiTZ1F4ZGp5btO/DlVaDrzCscP34JQzKe8DttCxjoeYkWpRK9Gi0882m2LHB0qBetbW1oV6uh7lChq2fQfYIZa7jTAbPRAL3BuBivxBcvX3kLximolU1QtnVCZ2CBWlkpJxaMevR0qqBsa4O6Uw2VSo3BUZ3r1KNp/ZSsSNgCHXw5DzCPGk+We7sROed24623V6Gwrifoh0CxCb8stWkKUQ8UajyaVvpHWcbb6cBETwOO7tuFiwXVgsmHf7SVHT8mxl/G7XUB++woivJy0Ni9tK56fmoIpXm3UKioRJdmIqRndot5hvpeSodQ+10pPjXvwPEcmB5qxy8/70Rq9hMEGP3cq/lOdFTnYfsPP6G8mUVu9DLJSIp5s010PS+K8cXHH+FYWiZK7ivwoKwMTapezAmCvdgx3FaJownHkXe/FOcTdmPdxv3QmGTMFGTwYkXc3BwYaCjE+tVrkJx+G6WlCpQ+KMPz1leuyO2LUHYTrib+C4npZXCv6nCiLvcXbIw7CtXrjWviti7WXfTlnbM8zY5jbVRk4F8/HkILQYDF5R585/B4+67xn5nL00FVfgVvv/W/+PVOtaxDEdg4f9VUiUtXs9ArOk7To15PTR4u3ql8PZ48uYBOp4NarUZ7e/sbu9rYZDoA/+wo7dnZ2aVGuFIOzOhGoVJ52qFCe/0D/LQnEZUtKqgW8dUYmzIu/rDxgJhMJrx8+XIZDx43ni2Q9gRbVqVSYWLCvX/A0w7PJ298ecoE8kmNJ8e3bXYYp7d/iL+s34a2IN92Md6xCb8ctYnKUA8UajyiZvqF8cXbYZ1H++M87NwRj/pufxt8hC584QlLyL+TwvKcIWwXvUpjT5MsFmvArzDlM3ozJaV0eDNs5Hul5h0ontWkR3FGKg6nZGBg0r3ZTz57YUnr3BRKrqVg/4kLGJgMLDKxmLfDbkFVdjI+/DQOtV1aWBbmXUF12NGX3v9mx16hKCcXja0q9PR0o+zqWWz6bDPUUzLWAngDcdIubnYTnmSl4B/rNqO+XwcW7GjBx/fIMtOLfV+tRXZ9v/uplt2IO8k/Ykv8OfRNuJ/Qitvq7Zpnc5Vz2jHa3YCjh39G9qNm4Y8Nb6AQpP1yC4HPSITk6TA7+AIb/vrf+CE5E5Myonfq+pXYv+Uz/G3NF3jY9Gr53hm7GTkXU1Chcu9h8dZjdHQU9fX1qK2tRV1d3Ru5AvXd0NCA6elp72a4NrqPD/ehuqoKVU/Z9QzPFDn48Yd45Fc8wzNXXhWePq3BgHZ62YTfYDCgqalpmQ48bjxbqLX0+B4eHhbp4L7ljS+fFfxkUuP5cecyT/U9x9er3sF3B6+AF+uRh8V4xyb8PIWIbdQDhRqPuLmScL55s1exZjSXZmLPwTNQj8zIfi3oG0/SPddAicV1FOHGaNWBmncgeOzoy4bSa0g8/itUg7oV/whkvs16De6cT8bJywWYMrkj/coZOgLe7HtVVYRd/1yLNeu/weXcUuhET+wdNgv625/gTOIB/Jx0GVrjPGYm+pCbdgQbNmzA+VvFGGdHAhL8Y0sDWp7kYvf3G7F63Ve4klOMZfv1HTaM9LUjO/0E1q7+Aqcu3UBOTg4yLpzDmd+uQK3RL/59ELRVxI9nY+sGDJMDSD9+GBkFNTAssA3AIoAQ3nK5hdBvpEHzdHA6ZnF21+dYvWk3Wgf1fpYyWKGqfgpFcQ4+/WA1jl1/AJNFeDrWzOALpJy6jAkfmx7Z0bKewFDs801cLAhVoH4Zb+E/FgnYHbV2MciV7iWSjl1Av0kU+IvtUxBWdr2B86UDjxvPFmh7Ai3v8b1cB3fDeONL1HRZt9R4fp06bWhRZOAvf3gbGZXdy/rLb/3XBRjv2IRfrloE5agHCjUeQRNlQfB5W9BYlo20q3nQ6MyyBjcfTxalxUKUWIugUZiIVh2oecvGc9ox3F6N8+czoNZMyRq3cofFvH4Y+Vd/RdaDBszbxP89+0YR8LZbMNyjRktLG9paW6Hq7MGcVTgRYkshTDM6aLUjGB2dwLzNDsuCCePjoxgbG4dWO4b5QNcV+aYGh90GbR/j0+ri067qguiwFNd64DnTNEa0WoyNj2F4eAiaoSGMjOlgEZ3pLWiryCfPZjPrUZZ9BVmKOsy83g8gqh7SWx63kDqOMHB/OrQ9uIR33/sAdx63LYvy7asppkkNTu7YiHU/JmFEb3IXcVjZ0TF4XpyJq4oWX9UiIs+fFoGQFGAFsIZfyocAT1SIZxMVDfstNTdqPH+C2ExjuJ68A3/++Af06oN/08p4xyb8/tQmtFMPFGo8wqZyofzxtlvN6O/qxKhudtnrRl/A/vB81ZHKo8SS8hEN+dGqAzVv2XhOK4ZfdUAzHtjSG7ljwWKcgErViVkfTyZ9Ycjm7auyjzxKPEosRpWHx7NZZifRqX4Fo1Xejygfsqwoi8dtRcBRVtmfDvphNXZuWoO9qdnQmfxvHncszEKRcQzvrvoMVS/dp09ZZjRgV2Z6OloGA1syGgo57ZZ56PR62OzCE2b8aREIFwHWdDeOHk2TdUqPlA8BnqgQzyYq6ro1TGnR29OD3r4+9Pf1YmBozO+pQ75w5OQFys0fJjUe3x/b39iIXV99gl1n8jDvOsWJX0PKynjHJvxS6oQgn3qgUOOFoMk+IeXwZmVcl08EYaYcPGEN6TtKLGkvkW+JVh2oecvHc4/XUPYs20cil4/ccnL5UuJRYjH+PDx/Np5drjbBlnuTvoPlHIp6/nSwzc2i+FIC1n+9Cy39E/6XXTnt6Korxvr338PJrGoADgy2VbuuS9fvQbeCCNb+2m81TeHx/XwoHlairrYaRfl30dStFT64YsG2mh/heGoGJgzCfT7+tPDn39suwJqfQl1NMwzCF3vexf2mBXii0mKbwz6PhkqF67jS+vpaKEoKUPXi5eLG67HeBiTujcflbAWaW57j2rljuFPZsmgXwa/oVsxtRWB+/t6sFFtcnx1VXFt0EevXf40KZf+KlokyHWITfrHCIbyP5oFHKUsk60DNjVK3cGJFqw7UvKnxwtWH1Lwp8SixmJ48PJ4tXH0h5SeSuUlxDkW+Px2YfbS7ETu+3ogTtyph8/uU04m58S4c3boOH21Lxsy8Ec9K7riuwsomLBAtTfOlxaCqHjczr2LvrgOo7RjA8/xTOJh2dzGKPGuLUTeEW6kHseHjn9A9KTy2158WvnxK5QmwnE6wM/5XskdFgCdyKraZhpW4k30DB37ai6IaNfoa83HgyEmMvP59M8OCC+7ejsfqMVgW5lDw6x7sPJsHo/8XOCLP/m/F3PzX4JegxuN5m53QIGnbl4g/fRuTRsuKlooy3rEJP09tYhv1QKHGI26uJBw1b0qQV4FVAAAHR0lEQVQ8SixJAaLAEK06UPOmxgtX11PzpsSjxGJ68vB4tnD1hZSfSOYmxTkU+XJ0cFhNeHw7FZ99uxd9OuFTcZ+c7CbcT0/A239ei8oWNXJvZbqupp5R4dN2n5WDz2SbZZ/knEbi+WxMGqaRefKgK6CkecE9WWNBvFprKqC4m4U93+9Eo0YvcCZHC0EFzg0lFnPDwxPbWGyC9qd3kZB0Gn2TRjy5fQq/XHsIw/y8K6jdy6e3Efd9PJp6htDRUI6UY6fQ3OuOts1pUlAmMbegQLwqUeN5QYuSTigfXME3/9yB6o4RVyRnUYGAbhnv2IQ/IMlWVph6oFDjrax18mtT86bEo8SSr0jklYxWHah5U+OFq6epeVPiUWIxPXl4PFu4+kLKTyRzk+Iciny5Opgme3B6XxxOZFYuPjHn8RlsLMQH7/wR2xPOICO7yHWNEp00JeXXbjMi7chuXC9phEHTjPgd+3CvogrdI+6N/BPdTbh44RKKC7OwdfMmPOoQnh0vVwsp/975lFgMl4cntjlsVty/dhwnLxfBaBrDqZ934VJuGdQ9Q7A6HLh/4Wf8nJqD/qEhNJffwb7E0+jSCQOhebdlJWkxt5Vg+dNhpdje9S36ASRu+wa/5VTBFHxAl0VIpkNswr8oR+gT0TrwqJWJZB2ouVFrFy68aNWBmjc1XrT2H6UOlFhMTx4ezxauvpDyE8ncpDiHIl++DnaMdVVjx9ZteKLyfea6Nz+rvg8Hv3kf//PXj5H9WOm6rKI4Kt7lKdIOmwnPKivQO26EdXYcj8se4nnbSxjmLLAYtFDkF6C1ux9DnbWI3/09btcPCtzK10JQzecNJRZzwMMT29hJXKqGCrT1auGwm9FYVY6qulZMzMzBsaDF8R1bUfjCHT1Z87wAX23eiuYgA0r5bLxXppiblymoJDWeTxJWM8oyU7D7+BVoZ2mCHjLesQm/T7VDk0k9UKjxQtPq5ajUvCnxKLGWtzx6cqJVB2re1HjhGgHUvCnxKLGYnjw8ni1cfSHlJ5K5SXEORX4gOrCN6+2P7+D4LxfRpTXw16Q7F6BI248PNv2E9mGj6wr1eUysLXa73T0mX6ftDhv04wPIvXIWSam3Mb1gQX/LUxzcuxPJl3Mx4bVwPRAt/PUFJRbzxcMT29g9W97kDuznhMNhd+0hmJuZwDNFFn7cuQc37t1HaUkBMq9l4pny1WJcDX/tCtQu5hZofXF5arxl+A4bOutKcCghBb2TphWt2/fGZrxjE35vRUKcph4o1Hghbv4iPDVvSjxKrMUGR2EiWnWg5k2NF66hQM2bEo8Si+nJw+PZwtUXUn4imZsU51DkB6qDdd6I5pqnaO0egs3PE/uJzmfILX6MWTtcVyj4+8d0wmazwGQywmSeg93hcEW6NpvNMJtMsHkFzgpUC55vSizmh4fHs3lzZE/+Xe02m2EyGmE0GjFvYQHvQvdTjBqbGs9bH5a2zU3jRfVTtPZo4fAzvsV1efeMd2zCz1OI2EY9UKjxiJsrCUfNmxKPEktSgCgwRKsO1Lyp8cLV9dS8KfEosZiePDyeLVx9IeUnkrlJcQ5FfuA6uJ8e22yvn6RzSDkdNlisNtdT0tBNKTkEAjQFroW0A0os5oWHx7NJMwyPhZobNZ5YBefriM+Uk31P/8Um/GK1Q3hPPVCo8ULYdAE0NW9KPEosQaOj7CZadaDmTY0XrmFAzZsSjxKL6cnD49nC1RdSfiKZmxTnUOTHdFhSlVILSizGkIfHsy217s2kqLlR44VLFcY7NuEPl9p+vjDB0IjmgRdMe6XqUOpAiSXFNxryo1UHat7UeOHqe2relHiUWExPHh7PFq6+kPITydykOIciP6bDkqqUWlBiMYY8PJ5tqXVvJkXNjRovXKow3rEJf7jU9vOFCYZGNA+8YNorVYdSB0osKb7RkB+tOlDzpsYLV99T86bEo8RievLweLZw9YWUn0jmJsU5FPkxHZZUpdSCEosx5OHxbEutezMpam7UeOFShfGOTfjDpbafL0wwNKJ54AXTXqk6lDpQYknxjYb8aNWBmjc1Xrj6npo3JR4lFtOTh8ezhasvpPxEMjcpzqHIj+mwpCqlFpRYjCEPj2dbat2bSVFzo8YLlyqMd2zCHy61/XxhgqERzQMvmPZK1aHUgRJLim805EerDtS8qfHC1ffUvCnxKLGYnjw8ni1cfSHlJ5K5SXEORX5MhyVVKbWgxGIMeXg821Lr3kyKmhs1XrhUYbxjE/5wqe3nCxMMjWgeeMG0V6oOpQ6UWFJ8oyE/WnWg5k2NF66+p+ZNiUeJxfTk4fFs4eoLKT+RzE2KcyjyYzosqUqpBSUWY8jD49mWWvdmUtTcqPHCpQrjHZvwh0ttP1+YYGhE88ALpr1SdSh1oMSS4hsN+dGqAzVvarxw9T01b0o8SiymJw+PZwtXX0j5iWRuUpxDkR/TYUlVSi0osRhDHh7PttS6N5Oi5kaNFy5VGO//ByWfSRl5dBcfAAAAAElFTkSuQmCC"
    }
   },
   "cell_type": "markdown",
   "id": "phantom-humidity",
   "metadata": {},
   "source": [
    "由于拿到的数据不含一类指标相关的数据，故暂只计算二类指标数据。根据二类指标的假设检验可以认为二类指标符合比例之差的单侧检验的类型，计算公式为：\n",
    "![image-3.png](attachment:image-3.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "retained-yacht",
   "metadata": {},
   "source": [
    "#### 数据读取与处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "turkish-circle",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 连接数据库读取\n",
    "conn = create_engine('mysql+pymysql://root:123456@localhost/pandas?charset=UTF8MB4')\n",
    "ad_data = pd.read_sql_query(\"select * from ab_data\",con=conn)\n",
    "\n",
    "# ad_data = pd.read_csv(r'F:\\【3】数据分析\\【7】指标体系与ABtest\\[2]模块二\\模块二作业\\ab_data.csv')\n",
    "# ad_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "organic-signal",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>group</th>\n",
       "      <th>landing_page</th>\n",
       "      <th>converted</th>\n",
       "      <th>date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>851104</td>\n",
       "      <td>2017-01-21 22:11:48.556739</td>\n",
       "      <td>control</td>\n",
       "      <td>old_page</td>\n",
       "      <td>0</td>\n",
       "      <td>2017-01-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>804228</td>\n",
       "      <td>2017-01-12 08:01:45.159739</td>\n",
       "      <td>control</td>\n",
       "      <td>old_page</td>\n",
       "      <td>0</td>\n",
       "      <td>2017-01-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>661590</td>\n",
       "      <td>2017-01-11 16:55:06.154213</td>\n",
       "      <td>treatment</td>\n",
       "      <td>new_page</td>\n",
       "      <td>0</td>\n",
       "      <td>2017-01-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>853541</td>\n",
       "      <td>2017-01-08 18:28:03.143765</td>\n",
       "      <td>treatment</td>\n",
       "      <td>new_page</td>\n",
       "      <td>0</td>\n",
       "      <td>2017-01-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>864975</td>\n",
       "      <td>2017-01-21 01:52:26.210827</td>\n",
       "      <td>control</td>\n",
       "      <td>old_page</td>\n",
       "      <td>1</td>\n",
       "      <td>2017-01-21</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id                   timestamp      group landing_page  converted  \\\n",
       "0   851104  2017-01-21 22:11:48.556739    control     old_page          0   \n",
       "1   804228  2017-01-12 08:01:45.159739    control     old_page          0   \n",
       "2   661590  2017-01-11 16:55:06.154213  treatment     new_page          0   \n",
       "3   853541  2017-01-08 18:28:03.143765  treatment     new_page          0   \n",
       "4   864975  2017-01-21 01:52:26.210827    control     old_page          1   \n",
       "\n",
       "         date  \n",
       "0  2017-01-21  \n",
       "1  2017-01-12  \n",
       "2  2017-01-11  \n",
       "3  2017-01-08  \n",
       "4  2017-01-21  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 处理数据类型\n",
    "ad_data[['user_id','converted']] =ad_data[['user_id','converted']].astype(int)\n",
    "\n",
    "# 转换日期\n",
    "ad_data[\"date\"] = ad_data.timestamp.str[:10]\n",
    "ad_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "european-ability",
   "metadata": {},
   "source": [
    "#### 计算 z₁-α 和 z₁-β"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "chinese-earth",
   "metadata": {},
   "source": [
    "根据常规默认值确定α，β，k的值：α=0.05，β=0.2，k=1 (组件样本均衡)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "expected-wonder",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.6448536269514722"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "0.8416212335729143"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "alpha = 0.05\n",
    "beta = 0.2\n",
    "k = 1\n",
    "z_alpha = stats.norm.ppf(1-alpha)\n",
    "z_beta = stats.norm.ppf(1-beta)\n",
    "\n",
    "display(z_alpha,z_beta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "killing-closure",
   "metadata": {},
   "source": [
    "#### 计算对照组点击率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "constitutional-methodology",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.1203863045004612"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p_control = ad_data.converted[(ad_data.group==\"control\") & (ad_data.landing_page==\"old_page\")].mean()\n",
    "p_control"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "interested-trustee",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.10589344218918344"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p_c = p_control*(1-p_control)\n",
    "p_c"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "impossible-plaintiff",
   "metadata": {},
   "source": [
    "#### 计算实验组点击率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "variable-flight",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.1303863045004612"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# H1：对照组点击率 - 实验组点击率 ＞ 0 ，为计算样本量，指定一个非0且偏小的值为0.01\n",
    "p_treatment = p_control + 0.01\n",
    "p_treatment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "worldwide-medicare",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.11338571609917422"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p_t = p_treatment*(1-p_treatment)\n",
    "p_t"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "confident-midwest",
   "metadata": {},
   "source": [
    "#### 计算样本量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "confirmed-dublin",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "24151.17184313226"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n_t = (p_c + p_t)*(((z_alpha + z_beta)/(p_c - p_t))**2)\n",
    "n_t"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bibliographic-lyric",
   "metadata": {},
   "source": [
    "#### 判断是否满足最低样本量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ready-screening",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "group      landing_page\n",
       "control    new_page          1928\n",
       "           old_page        145274\n",
       "treatment  new_page        145311\n",
       "           old_page          1965\n",
       "Name: user_id, dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 计算总体数据\n",
    "ad_data.groupby([\"group\",\"landing_page\"])[\"user_id\"].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "quantitative-knowing",
   "metadata": {},
   "source": [
    "实验组数据为量145311，计算样本量为24151，小于实验组数据量"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "colonial-privilege",
   "metadata": {},
   "source": [
    "### 实验检验"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "everyday-democrat",
   "metadata": {},
   "source": [
    "#### 计算统计量的显著性p值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "institutional-hindu",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6687"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "6700"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 抽取一天的数据量,统计样本量\n",
    "data = ad_data[ad_data.date==\"2017-01-08\"]\n",
    "n1 = data.converted[(data.group==\"control\") & (data.landing_page==\"old_page\")].count()\n",
    "n2 = data.converted[(data.group==\"treatment\") & (data.landing_page==\"new_page\")].count()\n",
    "display(n1,n2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "parental-converter",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.11888739344997756"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "0.12074626865671642"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "0.001858875206738858"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 计算对照组和实验组的点击率\n",
    "p1 = data.converted[(data.group==\"control\") & (data.landing_page==\"old_page\")].mean()\n",
    "p2 = data.converted[(data.group==\"treatment\") & (data.landing_page==\"new_page\")].mean()\n",
    "# 差值\n",
    "p_df = p2-p1\n",
    "\n",
    "display(p1,p2,p_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "armed-cooking",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0056134624424126955"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 计算方差\n",
    "var_ = np.sqrt((p1*(1-p1)/n1)+(p2*(1-p2)/n2))\n",
    "var_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "original-hierarchy",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.37026713217867524"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 计算显著性p值\n",
    "p = 1-stats.norm.cdf(p_df,0,var_)\n",
    "p"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "promotional-gauge",
   "metadata": {},
   "source": [
    "#### 用统计量的显著性P值与显著性α比较做决策"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "entire-japan",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "统计量的显著性p(0.37) > 显著性α(0.05),说明实验组点击率 ≤ 对照组点击率\n"
     ]
    }
   ],
   "source": [
    "if (p > alpha):\n",
    "    print(\"统计量的显著性p(%.2f) > 显著性α(%.2f),说明实验组点击率 ≤ 对照组点击率\" % (p,alpha))\n",
    "else:\n",
    "    print(\"统计量的显著性p(%.2f) ≤ 显著性α(%.2f),说明实验组点击率 > 对照组点击率\" % (p,alpha))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "understood-destination",
   "metadata": {},
   "source": [
    "#### 封装"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "bored-class",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ABtest_p(df:pd.DataFrame,group_col: str = None, value_col: str = None ,alpha:float = 0.05):\n",
    "    '''\n",
    "    :param df: 被分析DateFrame对象\n",
    "    :param alpha: 显著性\n",
    "    :param group_col: 组列的名字，默认为df的第一列\n",
    "    :param value_col: 值列的名字,默认为df的第2列\n",
    "    :return: tongjiliang p_value p_type\n",
    "    '''\n",
    "    # 列名\n",
    "    if not group_col:\n",
    "        group_col = df.columns[0]\n",
    "    if not value_col:\n",
    "        value_col = df.columns[1]\n",
    "        \n",
    "    # 设置输入参数\n",
    "    temp_p = df.groupby(group_col,as_index=False)[value_col].mean() # 计算点击率\n",
    "    temp_n = df.groupby(group_col,as_index=False)[value_col].count() # 计算样本量\n",
    "    values = temp_p.iloc[1,1] - temp_p.iloc[0,1] # 计算统计量\n",
    "    var_ = np.sqrt((temp_p.iloc[0,1]*(1-temp_p.iloc[0,1])/temp_n.iloc[0,1])\n",
    "                  +(temp_p.iloc[1,1]*(1-temp_p.iloc[1,1])/temp_n.iloc[1,1])) # 计算方差\n",
    "    p_value = 1-stats.norm.cdf(values,0,var_) # 计算显著性p值\n",
    "    if (p > alpha): # 判断显著性\n",
    "        result = \"无显著差异\"\n",
    "    else:\n",
    "        result = \"有显著差异\"\n",
    "    # 保存结果\n",
    "    test_result = pd.DataFrame({'H0':temp_p.iloc[0,1],'H1':temp_p.iloc[1,1],\n",
    "                                '方差':var_,'显著性p':p_value,'显著性检验':result},index = [0])\n",
    "    return test_result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "disabled-parliament",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>group</th>\n",
       "      <th>converted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>treatment</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>treatment</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>control</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>control</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>157</th>\n",
       "      <td>control</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         group  converted\n",
       "3    treatment          0\n",
       "17   treatment          1\n",
       "43     control          1\n",
       "66     control          0\n",
       "157    control          0"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 调用\n",
    "temp  = ad_data[ad_data.date=='2017-01-08'].loc[((ad_data.group==\"control\") & (ad_data.landing_page==\"old_page\")) \n",
    "                  | ((ad_data.group==\"treatment\") & (ad_data.landing_page==\"new_page\")),[\"group\",\"converted\"]]\n",
    "temp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "floral-winning",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>H0</th>\n",
       "      <th>H1</th>\n",
       "      <th>方差</th>\n",
       "      <th>显著性p</th>\n",
       "      <th>显著性检验</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.118887</td>\n",
       "      <td>0.120746</td>\n",
       "      <td>0.005613</td>\n",
       "      <td>0.370267</td>\n",
       "      <td>无显著差异</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         H0        H1        方差      显著性p  显著性检验\n",
       "0  0.118887  0.120746  0.005613  0.370267  无显著差异"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ABtest_p(temp)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
